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ABSTRACT 

We show that in cases of marginal detections (~ 3a), such as that of Baryonic 
Acoustic Oscillations (BAO) in cosmology, the often-used Gaussian approxima- 
tion to the full likelihood is very poor, especially beyond (~ 3a). This can 
radically alter confidence intervals on parameters and implies that one cannot 
naively extrapolate la error bars to 3a and beyond. We propose a simple fit- 
ting formula which corrects for this effect in posterior probabilities arising from 
marginal detections. Alternatively the full likelihood should be used for param- 
eter estimation rather than the Gaussian approximation of a just mean and an 
error. 

Subject headings: Methods: statistical, Cosmology, cosmological parameters, 
large-scale structure of Universe 
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Making observational detections in cosmology is hard. Two to three sigma evidence for 
new physics is common in the literature and the community is rightly skeptical of results with 
marginal statistical significance. Even in the case of 5a results, detections can be questioned 
if the results are surprising or conflicting data is released. 

When detections are expected there is a natural t endency to accept them more read- 
ily. Baryon Acoustic Oscillations (BAO) (BAO; e.g., iBassett fc Hlozekl 12010) are a good 
example. The o riginal BAO results of the SDSS and 2dF teams (jEisenstein et al.l 12005 



Cole et al.l 120051 ) were at less than 3<r significance. Nevertheless, the detection has been 
essentially unanimously accepted by the community (although see lSylos Labini et al.l 120091 ) 
despite the difficulty of lo calizing the BAO pea k, e. g., illustrated by the s hift in BAO results 
between 2007 and 2009 inlPercival et al.l (120071 ) and lPercival et al.l (120101 ) and recent studies 
of mock catalogs (IKazin et al.ll2010bl ) which suggest that the BAO peak would be invisible 
in at least 10% of SDSS DR7-sized samples. 

Such detections are easy to accept since the detected peak is precisely in the place where 
it was expected to be. This willingness to accept marginal detections carries two dangers. 
First, the signal may actually be pure statistical fluctuation and hence provide precise but 
inaccurate knowledge. Second, it can actively discourage publication of other studies which 
are apparently at odds with the 'detection'. 

In this letter, we point out that the possibility that the detection is not real, but just a 
statistical fluctuation due to noise, has a significant impact on use of the data for parameter 
estimation. This arises because the full posterior typically has very non-gaussian tails even 
if the likelihood is gaussian, as illustrated in Figure ([1]). Like a scorpion, the sting is in the 
tail. 

The prototypical example we have in mind is the use of the BAO peak for cosmology, 
but the principle is valid generally. While the current best BA O combined dete ction is 
at the ~ 4.9cr level from the SDSS, 6df and WiggleZ catalogues (IBlake et al.l 120111 ). lower 
significance detections will always mark the frontier of the subject as we push to higher 
redshifts and different samples. The search for BAO with the SDSS, 2QZ and 2SLAQ survey 
quasars illustrates our point. While they show a peak in the expected place (10 5/i~ 1 Mpc, 
they also show a (presumably) fake peak near 85/i _1 Mpc ( Sawangwit et al.l 120111 ) . A more 
standard example is the recent 6df survey which found 2.4<r evidence for BAO while the 
WiggleZ detection is 3.2<r at z = 0.6. This will continue with a number of BAO first 
detections still to come in the next few years: 



The first separate detections of the radial and transverse BAO peaks whi ch will yield 
H(z) and dj\{z) separately. There is a claim of detection of the radial BAO (jGaztanaga et al 
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Fig. 1. — An example of how the posterior probability distribution has extended non- 
Gaussian wings, for finite detection probability: A peak is detected at 99% confidence level 
(Pdetect = 0.99) at 9 = 3 ± 1, while the gaussian prior of 9 = ± 4 was assumed. The dashed 
curve shows the naive gaussian posterior for 9, which e.g., would suggest 9 = 7 is ruled out 
at 4cr level, while it is actually allowed at 99.5% confidence level. 
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20091 ). However this is somewhat c ontroversial and the a s sociated uncertaint y illus- 
trates the main points of this paper ( Miralda-Escudd l2009t Kazin et al.ll2010al ). 



The first detection of BAO in photometric redshift surveys. \j 

The first detection of BAO in cluster data. The current status is the 2-2. 5a evidence 



from the maxBCG cluster catalogue (JHutsill2010l ). 
The first detection of BAO in neutral hydrogen, HI. 



The first detection of BAO in the Lyman-a forest (McDonald &: Eisensteinll2007l ). This 
is a method that will be employed by both the BOSS and LAMOST surveys. 

The first z > 2 BAO detection with Lyman break galaxies. 



In addition, as future surveys progress, it will be tempting to split a given volume up into 
narrower redshift bins to provide more data points. In doing so it is standard to ignore the 
statistical significance of the BAO detections in forecasting the resulting constraints on dark 
energy, just as it has been standard to ign ore them in using curr ent BAO results: prominent 
exa mples include the W MAP7 analysis (IKomatsu et al.l l201ll ). the SDSS supernova sur- 
vey ( IKessler et al.ll2009i ) and the grid margin alisation component of the Supernova Legacy 
(SNLS) 3- year ana lysis (jSullivan et al.l 120 111 ), all of which use the gaussian approximation 
to SDSS BAO results results. 

As a toy model to illustrate the impact of ignoring the finite detection probability of the 
peak, consider the posterior probability derived from a BAO experiment, P (6 \d, detected) 
where d is the data, 6 are the parameters to be estimated {e.g., w ,w a ,flk) and detected 
signifies the assumption that the app arent peak i s not just noise, i .e., the underlying model 



is correct. Then the full posterior is (IPress 



19971 : iKunz et al.ll2007h : 



P{6\d) = P detect P(9\d, detected) + (1 - P dete ct)P(0), 



:i) 



where Pdetect is the statistical significance of the detection and P{6) is the prior probability 
for the parameters which coincides with P(9\d, noise), the knowledge gained if the apparent 
peak is assumed to be pure noise, since in this case there is no new information. One of 
the most important implications of this formula is that the resulting posterior is very non- 
gaussian, with catastrophic widening where the likelihood drops below the prior. An example 



1 Current surve ys such as Mega-Z lack the number density to reduce shot noise to a level where d etection 
is possible. DES (JThe Dark Energy Survey Collaboration! l2005h and PanSTARRS JCai et al.l l2009h should 
provide the first detections while LSST will provide exquisite results ( Zhan et al.ll2009h . 
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of this non-gaussian distribution is shown in Figure [TJ An important component of the full 
posterior is that it must be correctly normalized: J P(6\d)d6 = 1. 

Assuming that the prior is much wider than the measurement, i.e., P(8\d, detected) ^> 
P(9) near the peak of the likelihood, Figure ([2]) shows how the errorbars grow as one decreases 
the detection probability Pdetect- This is done by finding the 68.3%, 95.5%, and 99.7% regions, 
using the normalization Pdetect < 1 for a gaussian distribution. 

Let us look at the example of Pdetect = 0.99 (Figured]). Although the 1 and 2 — a error 
bars are essentially unaffected, these are of little interest in parameter estimation since any 
real conclusions must be supported at the 99.7% (3a) level or better. At this level, Figure 
(T5]) shows that there is a dramatic change: there are no constraints on the parameter at 
all at this significance, despite the likelihood P(0\d, detected) possibly claiming wonderful 
constraints. What is happening is that one is transitioning from the likelihood to the prior 
when one moves sufficiently far from the maximum likelihood value such that: 

P(6\d, detected) = 0- -Pdetect) p ^ „ (1 _ p^p^y (2 ) 

* detect 

Since the prior should vary with 9 much less rapidly than the likelihood, there is a point at 
which one's constraints are actually driven by the prior, not the data. The basic concept 
is simple: one cannot extrapolate data beyond its realm of validity, as illustrated by the 
figures. 

We can go further and show how the non-gaussian sting can radically change even 1 — a 
errors when combining constraints from different measurements. In Fig. fl3]) we show a 
toy model with two independent measurements of two parameters (Di,D 2 ) = (0.3,0.3) ± 
(0.1, 0.15) and (D\, D 2 ) = (0.7, 0.7)±(0.1, 0.15), which are different due to some unaccounted 
for systematic error. Furthermore, each measurement is based on a 3.6a detection, which 
causes a slight expansion of 1 and 2a constraints (blue solid contours, corresponding to 
Ax 2 = 2.3 and 6) with respect to the gaussian constraints (black dotted contours), as 
described in this letter (see Equation [3] below) . However, as we see on the second panel, 
combining the two constraints leads to radically different posteriors, with and without the 
assumption of gaussianity, i.e., even the non-gaussian la combined errors are much bigger 
than the gaussian ones, illustrating that the effects need not only limited be limited to > 3a 
results. 

The frequentist analog of this statement is that, for a finite detection confidence level, 
the function x 2 (#|cf) should asymptote to a plateau far away from its minimum, rather than 
growing indefinitely. The maximum difference between the plateau and the minimum is 
roughly the square of the signal-to-noise of the detection, S/N. Therefore, the gaussian 
(quadratic) approximation to the likelihood (x 2 ) breaks down when AxjL uss > (S/N) 2 . 
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Fig. 2. — This figure shows how 68.3%, 95.5%, and 99.7% confidence levels for the width of 
the posterior probability distribution (associated with 1, 2, and 3cr gaussian errors) rapidly 
expand with decreasing detection probability, Pdetect- 
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Fig. 3. — A toy example of how finite detection significance can affect even la combined 
constraints: On the left, the solid (dashed) contours show the non-gaussian (gaussian) 1 and 
2cr constraints based on two measurements of D\ and D2, which each have 3.6<r significance. 
The right panel shows the combined constraints, which are radically different with and 
without the assumption of gaussianity. 
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Fig. 4. — Left: (a) Power spectrum of a hypothetical galaxy survey, normalized to a 
smooth power spectrum without BAO oscillations (see text for details). Right: (b) Ax 2 
(= -2 log [likelihood]) for the position of the BAO peak in the linear power spectrum 
[k < 0.1 Mpc^ 1 ) for this survey. The red dotted curve shows the quadratic/gaussian ap- 
proximation which grows indefinitely, while the actual Ax 2 (black solid curve) saturates at 
~ (S/N) 2 = 13.1 for the detection. The blue dashed curve is an interpolation between the 
gaussian and the asymptotic regime that we suggest in Eq. ([3]) and is a good fit to the actual 
A X 2 . 
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As an example, in Figure (jlj) we consider the expected constraints on the BAO scale 
from a hypothetical galaxy survey with 10 6 galaxies and bias of 1.5, spread over a volume of 
0.8 Gpc 3 at small redshiftsQ In Figure (S^), we show the expected power spectrum+errors, 



norma lized to a smooth power spectrum without BAO oscillations, using the lEisenstein fc Hu 



( 119991 ) fit to the transfer function. The solid curve in Figure (jib) shows A\ 2 for fitting this 
spectrum with a different BAO (or sound horizon) scale, marginalizing over the amplitude of 
the oscillations. Here, we only include linear scales, conservatively defined as k < 0.1 Mpc~ ] 
and assume gaussian errors for A(k) = [k 3 P(k)/(2n 2 )] 



1/2 



While the gaussian approximation to the likelihood (dashed curve in Figure 0b), is a 
good approximation close to the minimum, it grows indefinitely, while the real Ax 2 saturates 
at ~ (S/N) 2 . In order to reflect this, we propose a simple analytic function to approximate 
the true difference between \ 2 and its minimum value: 

Ay 2 
AX 2 « *= -j- r (3) 

As shown in Figure (jib), this takes the quadratic shape of the gaussian approximation 
close to the minimum since the denominator is then negligible, but Eq. ([3]) guarantees that 
Ax 2 remains smaller than (S/N) 2 of the detection, far from its minimum, which limits the 
statistical power of low signal-to-noise detections in constraining parameters. While the 
interpolating function ([3]) is in good agreement with the actual A% 2 , we should note that it 
is only an approximation, and ideally one should use the full likelihood of the model fitting 
the data (e.g., the full galaxy power spectrum) for an accurate statistical analysis. 

Finally, Figure (J5j) demonstrates the effect of non-gaussian posteriors on cosmologi- 
cal constraints. Here, we compare the gaussian approximation to likelihoo d distribution 



for d istances to z = 0.2 and z = 0.35 in Sloan Digital Sky Survey (SDSS; IPercival et al. 



20101). with our expe ctation from Equation fl3]). Given that total S/N for BAO detection in 



Percival et al.l (2010|) is \/13.1 for two degrees of freedom, we see significant deviations from 



gaussian likelihoods beyond 99% confidence level q 



Conclusions - In modern cosmology, as in many other branches of science, there is a 
clear division of labour between theorists and experimentalists. The theorists often desire 
a simple and convenient data product that can be expressed in Gaussian form as a mean 



2 For this example, we use WMAP7+BAO+ifo cosmology (|Komatsu et al.ll201ll ). 

3 Figure (O can be dire ctly compared to Figure 4 in lPercival et al.l ( 2010 ). However, we should point out 



that IPercival et al.l (|201fl) underestimate the tail of the likelihood, as they ignore the (small) possibility of 



misidentifying the BAO peak in their mock realizations. 
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Fig. 5. — Contours of Ax 2 = -2 logflikelihood] = 2.3, 6.0, and 11.8 (corresponding to 68.3%, 
95.5%, and 99.7% gaussian probabilities) for di stances to z — 0.2 and 0.35, from BAO 
detection in SDSS7 galaxies (IPercival et al.ll2010l ). The dashed black contours are gaussian 
fits quoted in lPercival et al.l (120101 ) . while solid blue co ntours correspond to o ur non-gaussian 
expectation, based on Eq. ([3]) (compare to Figure 4 in lPercival et al.l (j2010h ). Although the 
68.3% (la) and 95.5% (2a) confidence contours are essentially unchanged, the effect of the 
finite probability of having detected the BAO peak has a profound effect on the 99.7% (3a) 
contour. 



- 11 - 



plus an error bar without using the full likelihood or redoing the initial analysis. This is 
exemplified by the many studies using the Gaussian approximation to the SDSS Baryon 
Acoustic Oscillation (BAO) distance. 

In the case of marginal detections, such as in the case of BAO cosmology or gravita- 
tional wave astronomy in the near future, this will not suffice, and the possibility that the 
detections are pure noise, even if small, have to be taken into account to get both precise 
and accurate results. The main impact of taking this into account, is to radically alter the 
relationship between confidence intervals. While 68% confidence intervals may be essentially 
unchanged, 95%, 99.7% and higher confidence intervals can be radically altered. In the 
Gaussian approximation, these intervals are all trivially related to each other, but in the 
case where the finite detection probability is included this relationship is broken. Given that 
confidence limits must exceed 3<r to be generally accepted as providing truly secure knowl- 
edge about Nature, it is important to include finite detection probabilities in cosmological 
statistical analyses lest one's posterior is stung by the tail of the scorpion. We provide an 
explicit and simple way to do this via Eq. fl3]) , which uses the signal-to-noise of the detection 
to remove this sting. 

A further concern might be that BAO analyses must assume a redshift-distance relation 
(typically ACDM) to compute the two-point correlation function in the first place, instead 
of recomputing £(r) at every point in the cosmic parameter space. For models close to 
ACDM, this only causes small errors. However, if one wants to compute 3 - 5cr constraints 
one now includes models - because of the detection effect discussed in this letter- that are 
far from ACDM and where the approximation of using ACDM to convert from redshift to 
distance might be very poor. This raises additional issues about the true constraints on 
cosmic acceleration from current BAO surveys. 



Finally, it is a well-known phenomenon (e.g., iHenrion fc Fischhofilll986l ) that experi- 
mentalists in all fields tend to underestimate the size of their error bars. This is particularly 
important here. Not only do the smaller error bars lead one to be over- confident about 
standard gaussian results, they are also likely to make one ignore the need for considering 
the finite-detection effects discussed here. For example, if one believes one has a 5 sigma 
detection of the BAO, then one could safely ignore the finite detection non-gaussianity. 
However, if one's error bars were too small by a factor of two that would make the non- 
gaussian corrections much larger. In the case of BAO, estimating error bars on the correla- 
tion f unction is very difficult . Ideally one should run a very large number of mock simulations 



[e.g., iTakahashi et al.ll2009l ). but in the absence of this it is common to use approximations 
such as jack-knife or log-normal simulations which may not give the correct answer. 

The simplest way to take care of all of this effect is to use the full P(k) power spectrum 
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measurements in Monte-Carlo Markov Chains, as is possible in e.g., CosmoMC. 
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